Existence and Regularity of a Nonhomogeneous Transition Matrix under Measurability Conditions
نویسندگان
چکیده
This paper is about the existence and regularity of the transition probability matrix of a nonhomogeneous continuous-time Markov process with a countable state space. A standard approach to prove the existence of such a transition matrix is to begin with a continuous (in t) and conservative matrix Q(t) = [qij(t)] of nonhomogeneous transition rates qij(t), and use it to construct the transition probability matrix. Here we obtain the same result except that the qij(t) are only required to satisfy a mild measurability condition, and Q(t) may not be conservative. Moreover, the resulting transition matrix is shown to be the minimum transition matrix and, in addition, a necessary and sufficient condition for it to be regular is obtained. These results are crucial in some applications of nonhomogeneous continuous-time Markov processes, such as stochastic optimal control problems and stochastic games, which motivated this work in the first place.
منابع مشابه
Large Deviations for a Class of Nonhomogeneous Markov Chains1 by Zach Dietz
Large deviation results are given for a class of perturbed nonhomogeneous Markov chains on finite state space which formally includes some stochastic optimization algorithms. Specifically, let {Pn} be a sequence of transition matrices on a finite state space which converge to a limit transition matrix P . Let {Xn} be the associated nonhomogeneous Markov chain where Pn controls movement from tim...
متن کاملExistence and Measurability of the Solution of the Stochastic Differential Equations Driven by Fractional Brownian Motion
متن کامل
Large Deviations for a Class of Nonhomogeneous Markov Chains
Large deviation results are given for a class of perturbed non-homogeneous Markov chains on finite state space which formally includes some stochastic optimization algorithms. Specifically, let {Pn} be a sequence of transition matrices on a finite state space which converge to a limit transition matrix P. Let {Xn} be the associated non-homogeneous Markov chain where Pn controls movement from ti...
متن کاملOn Symmetric Hyperbolic Boundary Problems with Nonhomogeneous Conservative Boundary Conditions
Symmetric hyperbolic systems with nonhomogeneous conservative boundary conditions are considered. Existence, uniqueness, and regularity are established for the boundary value problem and the initial-boundary value problem. This boundary problem is not strongly well-posed in L2, a loss of derivatives is observed. Minimal regularity assumptions on the coefficients of the operator are made.
متن کاملOn Existence of Limiting Distribution for Time-Nonhomogeneous Countable Markov Process
In this paper, sufficient conditions are given for the existence of limiting distribution of a nonhomogeneous countable Markov chain with time-dependent transition intensity matrix. The method of proof exploits the fact that if the distribution of random process Q = (Qt )t 0 is absolutely continuous with respect to the distribution of ergodic random process Q◦ = (Q◦t )t 0, then Qt law −→ t→∞ wh...
متن کامل